A Theoretical Analysis of Deep Neural Networks for Texture Classification

摘要

We investigate the use of Deep Neural Networks for the classification ofimage datasets where texture features are important for generatingclass-conditional discriminative representations. To this end, we first derivethe size of the feature space for some standard textural features extractedfrom the input dataset and then use the theory of Vapnik-Chervonenkis dimensionto show that hand-crafted feature extraction creates low-dimensionalrepresentations which help in reducing the overall excess error rate. As acorollary to this analysis, we derive for the first time upper bounds on the VCdimension of Convolutional Neural Network as well as Dropout and Dropconnectnetworks and the relation between excess error rate of Dropout and Dropconnectnetworks. The concept of intrinsic dimension is used to validate the intuitionthat texture-based datasets are inherently higher dimensional as compared tohandwritten digits or other object recognition datasets and hence moredifficult to be shattered by neural networks. We then derive the mean distancefrom the centroid to the nearest and farthest sampling points in ann-dimensional manifold and show that the Relative Contrast of the sample datavanishes as dimensionality of the underlying vector space tends to infinity.